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How  TRAs  Got  Started 


■  “Identify  each  case  in  which  a  major  defense  acquisition  program 
entered  system  development  and  demonstration  ...  into  which  key 
technology  has  been  incorporated  that  does  not  meet  the  technology 
maturity  requirement ...  and  provide  a  justification  for  why  such  key 
technology  was  incorporated  and  identify  any  determination  of 
technological  maturity  with  which  the  Deputy  Under  Secretary  of 
Defense  for  Science  and  Technology  did  not  concur  and  explain  how 
the  issue  has  been  resolved.”  National  Defense  Authorization  Act  for 
Fiscal  Year  2002 

■  “The  management  and  mitigation  of  technology  risk,  which  allows  less 
costly  and  less  time-consuming  systems  development,  is  a  crucial  part 
of  overall  program  management  and  is  especially  relevant  to  meeting 
cost  and  schedule  goals.  Objective  assessment  of  technology  maturity 
and  risk  shall  be  a  routine  aspect  of  DoD  acquisition.”  DoDI  5000.2, 
paragraph  3. 7. 2. 2 
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What  is  a  TRA? 


Systematic,  metrics-based  process  that  assesses  the  maturity  of 
Critical  Technology  Elements  (CTEs).  A  CTE  is  a  technology  that  is 
both: 

1)  Essential  for  (threshold)  system  performance,  and 

2)  New  or  novel,  or  used  in  a  new  or  novel  way 

Uses  Technology  Readiness  Levels  (TRLs)  as  the  metric 
Required  for  all  MDAP  &  MAIS 

A  TRA: 

■  Is  not  a  risk  assessment 

■  Is  not  a  design  review 

■  Does  not  address  system  integration 


Increasing  maturity 


Technology  Readiness  Levels  (TRLs)  for  Hardware 


1.  Basic  principles  observed  and  reported 

2.  Technology  concept  and/or  application 
formulated 


3.  Analytical  and  experimental  critical  function 
and/or  characteristic  proof  of  concept 

4.  Component  and/or  breadboard  validation  in  a 
laboratory  environment 


Component  and/or  breadboard  validation  in  a 
relevant 

System/subsystem  model  or  prototype 
lemonstration  in  a  relevant  environment 

System  prototype  demonstration  in  an 
operational  environment 


8.  Actual  system  completed  and  qualified  through 
test  and  demonstration 

9.  Actual  system  proven  through  successful 
mission  operations 


Increasing  maturity 


TRLs  for  Software 


1.  Basic  principles  observed  and  reported 

2.  Technology  concept  and/or  application 
formulated 

3.  Analytical  and  experimental  critical  function 
and/or  characteristic  proof  of  concept 

4.  Module  and/or  subsystem  validation  in  a 
laboratory  environment 

5.  Module  and/or  subsystem  validation  in  a  relevant 
environment 


Module  and/or  subsystem  validation  in  a  relevant 
end-to-c 

System  prototype  demonstration  in  an 
rerational  high  fidelity  environment 

Actual  system  completed  and  qualified  through 
test  and  demonstration  in  an  operational 
environment 

Actual  system  proven  through  successful 
mission-proven  operational  capabilities 


What  is  a  Critical  Technology  Element  (CTE) 


A  technology  element  is  “critical”  if  the  system  being  acquired 

depends  on  this  technology  element  to  meet  operational 
requirements  with  acceptable  development  cost  and 
schedule  and  with  acceptable  production  and  operation 
costs  and  if  the  technology  element  or  its  application  is 

either  new  or  novel. 

An  element  that  is  new  or  novel  or  being  used  in  a  new  or 
novel  way  is  critical  if  it  is  necessary  to  achieve  the 
successful  development  of  a  system,  its  acquisition,  or  its 

(threshold)  operational  utility. 


CTEs  may  be  related  to  hardware,  software,  manufacturing,  or  life  cycle  at 

the  subsystem  or  component  level 


What  is  Relevant  Environment 


A  relevant  environment  for  the  demonstration  of 
a  technology  is  a  set  of  test  conditions  that 
provide  confidence  that  skillful  application  of 
that  technology  to  an  item  (component, 
subsystem,  or  system)  will  support  the  required 
(threshold)  functionality  of  that  item  across  the 
full  spectrum  of  required  operational 
employments. 


Examples  of  Relevant  Environment 


■  Physical  Environment  -  Mechanical  Components, 
Processors,  Servers  and  Electronics;  Kinematic 
Environment;  Thermal  and  Heat  Transfer  Environment; 
Electrical  and  Electromagnetic  Environment;  Climatic 
Environment;  User  and  Use  Environment 

■  Logical  Environment  -  Software  (Algorithm)  Interfaces; 
Security  Interfaces 

■  Data  Environment  -  Data  Formats  and  Data  Bases; 
Anticipated  Data  Rates,  Data  Delay  and  Data 
Throughput;  and  Data  Packaging  and  Framing 

■  Security  Environment  -  Connection  to  firewalls;  Security 
Appliques;  Rates  and  Methods  of  Attack 
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TRL  6  Hardware 

_ Minimum  Maturity  at  Milestone  B _ 

■  Definition:  System/subsystem  model  or  prototype  demonstration 
in  a  relevant  environment. 

■  Description:  Representative  model  or  prototype  system  is  tested 
in  a  relevant  environment.  A  major  step  up  in  a  technology’s 
demonstrated  readiness.  Examples  include  testing  a  prototype  in 
a  high-fidelity  laboratory  environment  or  in  a  simulated,  such  as  a 
tri-mode  seeker  for  a  missile  in  a  “stimulated”  environment. 


Supporting  Information:  Results  from  laboratory 
testing  of  a  prototype  system  that  is  near  the  desired 
configuration  in  terms  of  performance,  weight,  and 
volume.  How  did  the  test  environment  differ  from  the 
operational  environment?  Who  performed  the  tests? 
How  did  the  test  compare  with  expectations?  What 
problems,  if  any,  were  encountered?  What  are/were 
the  plans,  options,  or  actions  to  resolve  problems 
before  moving  to  the  next  level?  8 


TRL  7  Hardware 

_ Minimum  Maturity  at  Milestone  C _ 

Definition:  System  prototype  demonstration  in  an  operational 
environment. 

Description:  Prototype  near  or  at  planned  operational  system. 
Represents  a  major  step  up  from  TRL  6  by  requiring  demonstration 
of  an  actual  system  prototype  in  an  operational  environment  (e.g., 
in  an  aircraft,  in  a  vehicle,  or  in  space).  Example:  testing  the 
prototype  system  (radar)  in  a  test-bed  aircraft. 

Supporting  Information  Results 
from  testing  a  prototype  system  in 
an  operational  environment.  Who 
performed  the  tests?  How  did  the 
test  compare  with  expectations? 

What  problems,  if  any,  were 
encountered?  What  are/were  the 
plans,  options,  or  actions  to  resolve 
problems  before  moving  to  the  next 
level? 


Example:  Establishing  the  Relevant  Environment 


■  Advanced  Seacraft 

■  Two  Contractors,  both  using  existing  hull  forms 

■  One  is  a  Commercial  Ferry,  in  use;  This  hull  uses  an  Al  alloy,  in 
lieu  of  steel 

■  What  constitutes  the  “environment”? 

■  Does  commercial  ferry  use  qualify  as  Operational  Environment? 
(TRL  7),  or  Relevant  environment  (TRL  6)? 

■  Must  know  the  technical  drivers  of  performance,  Loading, 
Speed,  Sea  State 
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Example:  Advanced  Field  Artillery  System  - 
High  rate  of  Fire  155mm  SPH 


High  rate  of  fire  implied  need  for  autoloader;  Auto  loader 
impossible  with  bagged  propellant 

Approaches 

■  Liquid  Propellant:  HAN-TEAN  mixture  was  a  new  or 
novel  chemical  mixture  to  use  as  propellant 

■  Solid  Propellant  (originally  “Unicharge”):  Conventional 
energetic  compounds,  bonded  into  “toilet  paper  roll” 
sized  packets  that  could  be  auto-loaded:  Not  new  or 
novel,  but  application  is  new  or  novel 
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The  TRA  Process,  2003-2005 


■  Technology  Readiness  Assessments  conducted  just  prior  to  Milestone  B 

■  Critical  Technology  Elements  selected  by  Program  Manager  and  Contractor 

■  Component  S&T  and  DUSD(S&T)  can  add  technologies  to  the  list 

■  Assessments  conducted  by  an  independent  panel  of  experts 

■  OSD  Policy  states  that  Technologies  should  be  “demonstrated  in  a  relevant 
environment”  (“TRL  6”) 

■  Establishing  the  criteria  for  “relevant  environment”  is  difficult  and  subtle 

■  Historically,  most  MDAP  and  MAIS  enter  Milestone  B  with  one  or  more  immature 
technologies 

■  Component  S&T  Organization  prepares  the  TRA  and  forwards  (normally 
through  Acquisition  Executive)  to  DUSD(S&T) 

■  DUSD(S&T)  concurs,  concurs  with  reservation,  or  does  not  concur 

■  Immature  technologies  are  addressed  in  various  ways 

■  Requirement  for  a  Technology  Maturation  Plan 

■  Often  an  Update  to  the  TRA  is  required 


TRAs  explain  the  maturity  of  the  technologies  and  are  input  to 

the  decision  at  Milestone  B 


Title  10  Requirement  for  Certification  by  AT&L 


NDAA  2006,  10  USC  §2366a 

Certification  required  before 
Milestone  B  or  Key  Decision 
Point  B  approval: 

(a)  CERTIFICATION.  A  major  defense 
acquisition  program  may  not  receive 
Milestone  B  approval,  or  Key 
Decision  Point  B  approval  in  the 
case  of  a  space  program,  until  the 
milestone  decision  authority 
certifies  that  - 

(2)  the  technology  in  the  program 
has  been  demonstrated  in  a 
relevant  environment; 

•  •  • 

(10)  the  program  complies  with  all 
relevant  policies,  regulations 
and  directives  of  the  Department 
of  Defense 


tLonuniM 

1C.HMUi.Qt<r 
MM)  iJbUriCt 


’THE  UNDER  SECRETARY  OF  DEFENSE 
3010  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3010 


MAY  0  2  2006 


MEMORANDUM  FOR  SEE  DISTRIBUTION 


SUBJECT  Implementation  of  Section  2363a  of  Title  10.  Unites  States  Code 


Section  2366a  of  title  10.  United  States  Code,  as  enacted  try  section  SOI  of  the  National 
Defense  Authonzanou  Act  for  Fiscal  Year  2006  (Pub  L.  No.  109-163).  requires  the  Milestone 
Decision  Authority  (MDA)  for  a  Major  Defense  Acquisition  Program  (MDAP)  to  make 
certain  certifications  pnor  to  Milestone  3  or  Key  Decision  Pom:  B  approval 

To  fulfill  this  requirement,  die  MDA.  without  the  authority  to  delegate,  shall  sign  a 
memorandum,  subject  ’Program  Certification. '  prior  to  signing  the  Acquisition  Decision 
Memorandum  (ADM)  This  certification  memorandum  shall  t«  prepared  ’for  the  record." 
and  shall  include  the  statements  in  the  attachment  without  modification.  If  the  program  is 
initiated  at  a  later  decision  point,  e  g..  Milestone  C.  a  similar  memorandum  shall  he 
prepare!  as  a  matter  of  policy,  consistent  with  the  mrenr  of  the  starjte  The  certification 
memorandum  shall  be  submitted  to  the  congressional  defense  committees,  as  defined  at  10  U 
S.C- 101_(16).  with  the  first  Selected  Acquisiuon  Report  for  the  program  after  complenon  of 
the  certification 

The  MDA  may  waive  one  oc  more  of  components  (1)  through  (6)  of  the  required 
certification  (specifically,  one  or  more  of  paragraphs  (1)  through  (6)  in  the  attachment)  for  an 
MDAP  if  the  MDA  determines  that,  but  for  such  a  waiver,  the  Department  would  be  unable 
to  meet  critical  national  security  objectives  The  MBA  shall  submit  the  waiver,  the 
determination,  and  reasons  for  the  determination,  m  writing,  to  the  congressional  defense 
committees  within  3C  days  of  authorizing  the  waiver.  The  MDA  may  not  delegate  this 
waiver  authority . 

In  addition  to  the  certification  memorandum,  the  MDA  will  include  the  following 
statement  in  the  ADM  T  ha\-e  reviewed  the  program  and  have  made  the  certifications 
required,  or  executed  a  waiver  as  authorize!  by  secnon  2366a  of  title  10.  United  States 
Code  ’ 

This  policy  shall  apply  to  MDAPs  approved  by  me  and  to  MDAPs  managed  by 
Department  of  Defense  C  omponenr  Acquisition  Executives  or  the  Assistant  Secretary 
of  Defense  for  Networks  and  Information  Integration.  This  requirement  went  into  effect 
January  6.  20C6.  and  shall  be  reflected  m  the  next  revision  to  Department  of  Defense 
Instruction  50002 


Attachment: 
As  stated 


But  Waivers  Are  Allowed 


(c)  WAIVER  FOR  NATIONAL  SECURITY.  The 

milestone  decision  authority  may  waive  the 
applicability  to  a  major  defense  acquisition  program  of 
one  or  more  components  (as  specified  in  paragraph 
(1),  (2),  (3),  (4),  (5),  (6),  (7),  (8),  or  (9)  of  subsection 
(a))  of  the  certification  requirement  if  the  milestone 
decision  authority  determines  that,  but  for  such  a 
waiver,  the  Department  would  be  unable  to  meet 
critical  national  security  objectives. 


Whenever  the  milestone  decision  authority  makes  such  a 
determination  and  authorizes  such  a  waiver,  the  waiver,  the 
determination,  and  the  reasons  for  the  determination  shall  be  submitted 
in  writing  to  the  congressional  defense  committees  within  30  days  after 
the  waiver  is  authorized. 
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Potential  Impacts  of  10  USC  §2366a 


■  In  practice,  since  law  enacted,  no  waivers  have  been  granted. 

■  Critical  Technology  Elements  need  to  be  identified  at  earliest 
opportunity,  and  balanced  against  requirements  &  schedule. 

■  Importance  of  Test  &  Evaluation  in  pre  SDD  increased. 

■  Technology  Maturation  prior  to  MS  B  decision. 

■  Technology  Development  Strategies  aggregating  technology 
transitions  into  “blocks”. 
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Summary 


■  The  law  requiring  certification  of  “demonstration  in  a 
relevant  environment”  has  implications  for  both 
technology  developers  as  well  as  PMs 

■  Developers  of  new  technology  will  need  to  think 
ahead  to  “relevant  environments”  and  guide  their 
measurement  efforts  with  an  eye  to  easing 
incorporation  of  their  technology  in  system 
development. 

■  The  TRA  process  has  increased  the  interactions 
between  S&T  and  Acquisition  in  DoD.  A  TRA 
Deskbook  facilitates  the  process 
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